Parallel Scheduling Attention Mechanism: Generalization and Optimization

Over the past few years, attention is shining in field of deep learning, especially domain natural language processing (NLP). Its impressive effectiveness, along with ubiquitous implementations, have aroused our interest efficiently scheduling data-flow corresponding computations onto architectures many computing units to realize parallel computing. In this paper, based on manually analyzing optimum solutions for small instances, which are obtained by a satisfiability checking (SAT) solver, we propose general solution parallelize layers that widely adopted recent learning models. According solution, proposed hardware system m elements (PEs) connected unidirectional ring, m-time speed up achievable. For two specific application schemes attention, respectively recognize almost 25% and 50% original become redundant under those certain circumstances. To avoid unnecessary gains latency, come strategies optimization accordingly, further lead another solutions. By avoiding redundancy, adoptions optimized able additionally bring near reduction execution cycles, schemes. prove correctness these solutions, mathematically revealed their validity, as well utilized SAT solver conduct verification adopting themselves additional constraints formulated problems.